Surrounding assessment for heat map visualization

ABSTRACT

A method for assessing a surrounding area for heat map generation includes determining a location of a user and receiving historical information for the location of the user. The method also includes identifying a plurality of items in a surrounding area of the user, where the surrounding area includes at least a line of sight of the user as captured by an augmented reality device associated with the user. Analyzing the surrounding area of the user based on the plurality of items and the historical information to produce a plurality of risk levels for the plurality of items, where each risk level from the plurality of risk levels is associated with each item. Responsive to receiving sensory data for the user, the method also includes generating a heat map for the surrounding area of the user and display the heat map to the user in the augmented reality device.

BACKGROUND

This disclosure relates generally to augmented reality, and inparticular to assessing an area surrounding a user for generating a heatmap visualization.

Presently, manufacturing and warehouse floors utilize various forms ofautomation and robotics in conjunction with human operators tomanufacture and handle products. Though automation and robotics utilizevarious sensors programmed to provide safe operations in a givenenvironment, the human operators have to remain vigilant in the givenenvironment to provide another layer of protection. The products handledthrough automation and robotics can vary day-to-day and hour-by-hour,thus resulting in a constantly changing environment for the humanoperator.

SUMMARY

Embodiments in accordance with the present invention disclose a method,computer program product and computer system for assessing a surroundingarea for heat map generation, the method, computer program product andcomputer system can, responsive to determining a location of a user,receive historical information for the location of the user. The method,computer program product and computer system can identify a plurality ofitems in a surrounding area of the user, wherein the surrounding areaincludes at least a line of sight of the user as captured by anaugmented reality device associated with the user. The method, computerprogram product and computer system can analyze the surrounding area ofthe user based on the plurality of items and the historical informationfor the location of the user to produce a plurality of risk levels forthe plurality of items, wherein each risk level from the plurality ofrisk levels is associated with each item from the plurality of items.The method, computer program product and computer system can, responsiveto receiving sensory data for the user, generate a heat map for thesurrounding area of the user, wherein the heat map is based on theplurality of risk factors for the plurality of items in the surroundingarea. The method, computer program product and computer system candisplay the heat map to the user in the augmented reality device.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

FIG. 1 is a functional block diagram illustrating a distributed dataprocessing environment, in accordance with an embodiment of the presentinvention.

FIG. 2 is a flowchart depicting operational steps of a surroundingassessment program for assessing a surrounding area for generating aheat map visualization, in accordance with an embodiment of the presentinvention.

FIG. 3A illustrates a first-person perspective view of a heat mapvisualization that a surrounding assessment program generates for asurrounding area, in accordance with an embodiment of the presentinvention.

FIG. 3B illustrates an overhead view of a heat map visualization that asurrounding assessment program generates for a surrounding area, inaccordance with an embodiment of the present invention.

FIG. 4 is a block diagram of components of a computer system, such asthe server computer of FIG. 1, in accordance with an embodiment of thepresent invention.

FIG. 5 depicts a cloud computing environment, in accordance with anembodiment of the present invention.

FIG. 6 depicts abstraction model layers, in accordance with anembodiment of the present invention.

DETAILED DESCRIPTION

Embodiments of the present invention assess a surrounding area at alocation of a user and generate a heat map for risk levels associatedwith multiple objects in the surrounding area. Responsive to determiningthe location of the user, embodiments of the present invention receivehistorical information for the location of the user and identify itemsin the surrounding area of the user. Embodiments of the presentinvention analyze the surrounding area of the user and receive sensorydata for the user to generate a personalized heat map for thesurrounding area of the user. Embodiments of the present inventiondisplay the heat map for the surrounding area in a device associatedwith the user and responsive to determining a safe movement path throughthe heat map, embodiments of the present invention display the safemovement path through the heat map for the surrounding area in thedevice associated with the user.

FIG. 1 is a functional block diagram illustrating a distributed dataprocessing environment, generally designated 100, in accordance with oneembodiment of the present invention. The term “distributed” as usedherein describes a computer system that includes multiple, physicallydistinct devices that operate together as a single computer system. FIG.1 provides only an illustration of one implementation and does not implyany limitations with regard to the environments in which differentembodiments may be implemented. Many modifications to the depictedenvironment may be made by those skilled in the art without departingfrom the scope of the invention as recited by the claims.

Distributed data processing environment includes server computer 102,client device 104, and augmented reality (AR) device 106 allinterconnected over network 108. Server computer 102 can be a standalonecomputing device, a management server, a web server, a mobile computingdevice, or any other electronic device or computing system capable ofreceiving, sending, and processing data. In other embodiments, servercomputer 102 can represent a server computing system utilizing multiplecomputers as a server system, such as in a cloud computing environment.In another embodiment, server computer 102 can be a laptop computer, atablet computer, a netbook computer, a personal computer (PC), a desktopcomputer, a smart phone, or any programmable electronic device capableof communicating with client device 104, AR device 106, and othercomputing devices (not shown) within the distributed data processingenvironment via network 108. In another embodiment, server computer 102represents a computing system utilizing clustered computers andcomponents (e.g., database server computers, application servercomputers, etc.) that act as a single pool of seamless resources whenaccessed within the distributed data processing environment. Servercomputer 102 includes server-side surrounding assessment program 110Aand database 112. Server computer 102 may include internal and externalhardware components, as depicted and described in further detail withrespect to FIG. 4.

Client device 104 can be a laptop computer, a tablet computer, a smartphone, smart watch, a smart speaker, or any programmable electronicdevice capable of communicating with various components and deviceswithin the distributed data processing environment (e.g., servercomputer 102 and AR device 106), via network 108. Client device 104 maybe a wearable computer. Wearable computers are miniature electronicdevices that may be worn by the bearer under, with, or on top ofclothing, as well as in or connected to glasses, hats, or otheraccessories. Wearable computers are especially useful for applicationsthat require more complex computational support than merely hardwarecoded logics. In general, client device 104 represents one or moreprogrammable electronic devices or combination of programmableelectronic devices capable of executing machine readable programinstructions and communicating with other computing devices (not shown)within the distributed data processing environment via a network, suchas network 108. In one embodiment, client device 104 represents one ormore devices associated with a user. Client device 104 includes aninstance of user interface 120 for interacting with surroundingassessment program 110A on server computer 102 and surroundingassessment program 110B on AR device 106.

AR device 106 represents a user wearable augment reality device (e.g.,electronic contact lens, wearable electronic headset) with integratedmicrocircuitry capable of displaying content to the user. AR device 106includes microcontroller 114, display 116, sensor 118, where surroundingassessment program 110B operating on microcontroller 114 manages anddisplays content in display 116. Microcontroller 114 can include adisplay control circuit for display 116, a communication and powerconversion circuit for communicating via network 108 and managing anintegrated power supply, and a sensor readout and control circuit formonitoring eye movement of the user wearing AR device 106. Display 116allows for a user to view a heat map for a surrounding area,navigational instructions, menu items, and any other type of contentthat surrounding assessment program 110A and 110B can provide to theuser of AR device 106. In an example where AR device 106 is anelectronic contact lens, display 116 is a semitransparent display andmicrolens array is integrated into AR device 106 for viewing content. ARdevice 106 can also include a power storage module, a solar cell modulefor charging the power storage module, a biosensor module for collectingdata (e.g., tracking eye movement), and a communications and powermodule for communicating with server computer 102 and client device 104via network 108. Sensors 118 (e.g., biosensor module) collect variousdata for the user of AR device 106, where surrounding assessment program110 utilizes the data collected by sensors 118 to generate a heat mapfor a surrounding area. Sensors 118 can include one or more of: anaccelerometer, a barometer, a gyroscope sensor, a heart rate sensor, andan orientation sensor.

Network 108 can be, for example, a telecommunications network, a localarea network (LAN), a wide area network (WAN), such as the Internet, ora combination of the three, and can include wired, wireless, or fiberoptic connections. Network 108 can include one or more wired and/orwireless networks capable of receiving and transmitting data, voice,and/or video signals, including multimedia signals that include voice,data, and video information. In general, network 108 can be anycombination of connections and protocols that will supportcommunications between server computer 102, client device 104, AR device106, and other computing devices (not shown) within the distributed dataprocessing environment.

Surrounding assessment program 110 generates a heat map based on variousrisk factors for an area surrounding a user of AR device 106.Surrounding assessment program 110 determines a location (e.g., factoryor warehouse floor) for a user associated with AR device 106, where insome embodiments AR device 106 is paired to client device 104.Surrounding assessment program 110 receives historical information forthe location of the user and identifies items in a surrounding area ofthe user. Surrounding assessment program 110 analyzes the surroundingarea of the user and receives sensory data for the user from sensors 118on AR device 106. Based on an analysis of the surrounding area and thesensory data for the user, surrounding assessment program 110 generatesthe heat map for the surrounding are of the user and displays the heatmap for the surrounding area in client device 104 and/or AR device 106.Surrounding assessment program 110 determines a path through the heatmap for the surrounding area of the user and displays the path throughthe heat map for the surrounding area in client device 104 and/or ARdevice 106.

Database 112 is a repository that stores various data including riskfactors for known objects (e.g., manufacturing robotics), floorplans(e.g., manufacturing floor, warehouse floor), operational specificationsfor each known object (e.g., swooping radius of a robotic arm),historical operational data for each object, historical operation datafor a user of AR device 106, and any other data pertinent to surroundingassessment program 110 for generating a heat map for a surrounding arearelative to the user of AR device 106 and determining navigationalinstructions through the heat map. In the depicted embodiment, database112 resides on server computer 102. In another embodiment, database 112may reside on client device 104 or elsewhere within the distributed dataprocessing environment provided surrounding assessment program 110 hasaccess to database 112. A database is an organized collection of data,where database 112 can be implemented with any type of storage devicecapable of storing data and configuration files that can be accessed andutilized by surrounding assessment program 110, such as a databaseserver, a hard disk drive, or a flash memory.

User interface 120 enables a user to make requests of or issue commandsto server computer 102, client device 104, and AR device 106 via network108. User interface 120 also enables the user to receive information andinstructions in response on client device 104 via network 108. In oneembodiment, a user of client device 104 accesses user interface 120 viavoice commands in natural language. In one embodiment, user interface120 may be a graphical user interface (GUI) or a web user interface(WUI) and can display text, documents, web browser windows, useroptions, application interfaces, and instructions for operation, andinclude the information (such as graphic, text, and sound) that aprogram presents to a user and the control sequences the user employs tocontrol the program. User interface 120 enables a user of client device104 to interact with each instance of surrounding assessment program110A and 110B operating on server computer 102 and AR device 106,respectively.

FIG. 2 is a flowchart depicting operational steps of a surroundingassessment program for assessing a surrounding area for generating aheat map visualization, in accordance with an embodiment of the presentinvention.

Surrounding assessment program 110 determines (202) a location of auser. Surrounding assessment program 110 determines the location of theuser based on location information received from a device associatedwith the user (e.g., AR device, client device). Surrounding assessmentprogram 110 allows for the user to establish privacy settings whichdefine when surrounding assessment program 110 receives locationinformation for the user. In some embodiments, surrounding assessmentprogram 110 continuously receives location information from the deviceassociated with the user. In other embodiments, surrounding assessmentprogram 110 receives location information for the user at specificpoints in time and/or set intervals in time (e.g., work hours 9 am-5 pm,Monday through Friday), as defined by the user. Surrounding assessmentprogram 110 can also receive location information for the usersubsequent to determining that the user entered a given area (e.g.,warehouse or factory floor), based on the device associated with theuser connecting to a local network within the given area. The given areacan include multiple subareas, where each subarea from the multiplesubareas can include various risk factors for generating a heat map forthe surrounding area of the user.

In one embodiment, surrounding assessment program 110 determines a userwith an associated device is entering a defined work area (e.g.,manufacturing floor) based on an AR device associated with the userpairing to a local network in the defined work area. Surroundingassessment program 110 receives location information from the AR deviceand determines a location of the user within the defined work area basedon the received location information. As long as the AR device isconnected to the local network and/or the location information from theAR device is for a position within the given work area, surroundingassessment program 110 can continuously receive the location informationfrom the AR device associated with the user. In another embodiment,surrounding assessment program 110 utilizes a user defined time intervalof Sam-6 pm for determining a location of the user, where the userdefined time interval represents work hours for the user. Surroundingassessment program 110 receives the location information from a deviceassociated with the user during the defined time interval and determinesa location of the user.

Surrounding assessment program 110 receives (204) historical informationfor the location of the user. Historical information for the locationcan include various information associated with the determined locationand/or the user, for determining various risk factor for items withinthe determined location. Historical information for the user can includeoperational qualifications for operating one or more machinery items atthe determined location. Historical information for the determinedlocation can include a location for specific machinery in the determinedlocation, a list of manufacturers specified risks for specificmachinery, specified walking areas, specified risk areas, historicallyidentified walking areas, locations for incident reports, andcrowdsourced user defined risk areas. The location for specificmachinery in the determined location can include a location of an areafor a footprint of the specific machinery, dimensions for the specificmachinery, an operational area for the specific machinery, and anoperational volume for the specific machinery. The operational area andthe operational volume represent a space required for the specificmachinery to operate (e.g., a sweeping area of a robotic arm) withoutinterference from other objects and operating users. In someembodiments, the operational area and the operational volume can differfor the specific machinery based on a component being handled by thespecific machinery. Surrounding assessment program 110 has the abilityto source dimensions for a specific machinery, an operational area forthe specific machinery, and an operational volume for the specificmachinery directly from a manufacturer.

The list of manufacturers specified risks for specific machinery caninclude portions of the specific machinery to avoid contacting to avoidinjury, portions of the specific machinery to avoid contacting to avoidcontamination, and a list of safe equipment for operating the specificmachinery. Specified walking areas represent defined areas for the userto travel at the determined location without interfering with specificmachinery at the determined location. Specified risk areas representdefined areas for the user to avoid at the determined location to avoidinterfering with specific machinery at the determined location.Historically identified walking areas represent an overlay on thespecified walking area for locations where surrounding assessmentprogram 110 previously received location information from devicesassociated with multiple users traveling at the determined location.Locations for incident reports represent one or more event occurrences(e.g., unexpected movement from a specific machinery) at the determinedlocation for the user. Crowdsourced user defined risk areas representuser defined define areas to approach with caution at the determinedlocation to avoid potentially interfering with specific machinery at thedetermined location. Surrounding assessment program 110 can receive thecrowdsourced user defined risk area when displaying a heat map to theuser for a surrounding area at the determined location, discussed indetail with regard to (214).

Surrounding assessment program 110 identifies (206) items in asurrounding area of the user. Surrounding assessment program 110identifies items in a surrounding area of the user based on thedetermined location of the user and items captured by a device (e.g., ARdevice) associated with the user. In one embodiment, surroundingassessment program 110 utilizes a surrounding area representing an areain a line of sight as captured by a forward-facing camera on an ARdevice, where the line of sight of the camera represents the line ofsight of the user associated with the AR device. In another embodiment,surrounding assessment program 110 utilizes a surrounding arearepresenting an area in a line of sight as captured by a forward-facingcamera on an AR device and a surrounding vicinity of the user (e.g.,radius of 3 meters). The surrounding vicinity of the user of the userincludes areas not captured by the forward-facing camera on the ARdevice (e.g., behind the user) and allows for surrounding assessmentprogram 110 to identify items not in the line of sight of the user.

Based on the determined location of the user, surrounding assessmentprogram 110 identifies stationary known items in a surrounding area ofthe user and identifies mobile known items in the surrounding area ofthe user. Surrounding assessment program 110 has the ability to utilizeknown floorplans of the determined location (e.g., warehouse floor,manufacturing floor) to identify where stationary items are positionedin the floorplan relative to the determined location of the user.Furthermore, surrounding assessment program 110 has the ability toutilize known travel paths of mobile items within the floorplan of thedetermined location to identify where the mobile items travel relativeto the determined location of the user. Surrounding assessment program110 also identifies items in a surrounding area by receiving an imagefrom a forward-facing camera on an AR device associated with the user,where the image represents the line of sight of the user. Surroundingassessment program 110 identifies the items in the image utilizingobject recognition software and can compare the identified items in theimage to any identified stationary and/or mobile items relative to thedetermined location of the user. In other embodiments, surroundingassessment program 110 can communicate with various Internet of Things(IoT) devices within the surrounding area to identify items in thesurrounding area of the user, where surrounding assessment program 110can receive location information for the IoT devices and/or imagescaptured of the surrounding area of the user.

Surrounding assessment program 110 analyzes (208) the surrounding areaof the user. In this embodiment, surrounding assessment program 110analyzes the surrounding area of the user based on risk factors for theidentified items in the surrounding area of the user. For each of theidentified items, surrounding assessment program 110 determines a riskfactor relative to the user with the associated AR device based on thedetermined location of the user and the received historical informationfor the location of the user. For each of the identified items,surrounding assessment program 110 determines a distance between aspecific identified item and the determined location of the user.Surrounding assessment program 110 parses the received historicalinformation for the location of the user to identify a portion ofhistorical information relevant to the specific identified item. Aspreviously discussed, historical information for the user can includeoperational qualifications for operating one or more machinery items atthe determined location and historical information for the determinedlocation can include a location for specific machinery in the determinedlocation, a list of manufacturers specified risks for specificmachinery, specified walking areas, specified risk areas, historicallyidentified walking areas, locations for incident reports, andcrowdsourced user defined risk areas. Subsequent, to identifying theportion of historical information relevant to the specific identifieditem, surrounding assessment program 110 determines a risk score for thespecific identified item relative to the determined location of theuser.

In one example, surrounding assessment program 110 determines that apreviously identified item is machinery AB for assisting in moving loadson a factory floor is located 1.5 meters from the user based on a knownstationary location for machinery AB and the determined location of theuser. Surrounding assessment program 110 parses the received historicalinformation for the location of the user and identifies a portion of thehistorical information relevant to the machinery AB. Based on theportion of the historical information relevance to machinery AB,surrounding assessment program 110 determines that machinery AB has anoperational area positioned on a central axis in a shape of a circlewith a radius of 2 meters. Furthermore, surrounding assessment program110 determines that machinery AB has a specified risk area positioned ona central axis in a shape of a circle with a radius of 2 meters, wherethe specified risk area represents a defined area for the user to avoidat the determined location to avoid interfering with specific machinery(i.e., machinery AB) at the determined location. Based on the user beinglocated within the operational area of machinery AB and a specified riskarea, surrounding assessment program 110 determines that machinery ABposes a high risk to the user at the determined location of the user.

In another example, surrounding assessment program 110 determines that apreviously identified mobile item is machinery AC for assisting inmoving loads on a warehouse floor is located 6 meters from the userbased on a known location for machinery AC at a specific time and thedetermined location of the user. Surrounding assessment program 110parses the received historical information for the location of the userand identifies a portion of the historical information relevant to themachinery AC. Based on the portion of the historical informationrelevance to machinery AC, surrounding assessment program 110 determinesthat machinery AC is a mobile item that travels along a set path.Furthermore, surrounding assessment program 110 determines the set pathcrosses a specified walking area, where the user has historicallyutilized (i.e., historically identified walking areas). Based on theuser being in a specified walking area that crosses the set path ofmachinery AC, surrounding assessment program 110 determines thatmachinery AC poses a moderate risk to the user at the determinedlocation of the user. Furthermore, if surrounding assessment program 110determines the user is walking at the determined location and isexpected to cross paths with the set path of machinery AC (i.e.,collision), surrounding assessment program 110 can elevate the moderaterisk to a high risk since the user is expected to cross paths withmachinery AC as it moves across the set path where the user will belocated.

In yet another example, surrounding assessment program 110 determinesthat a previously identified item is machinery AD located on a factoryfloor 3 meters from the user based on a known stationary location formachinery AD and the determined location of the user. Surroundingassessment program 110 parses the received historical information forthe location of the user and identifies a portion of the historicalinformation relevant to the machinery AD. Based on the portion of thehistorical information relevance to machinery AD, surrounding assessmentprogram 110 determines that machinery AD has an operational areapositioned on a central axis in a shape of a circle with a radius of 2meters. Furthermore, surrounding assessment program 110 determines thatmachinery AD has multiple crowdsourced user defined risk areas, wherethe multiple crowdsourced user defined risk areas define an areapositioned on a central axis of machinery AD in a shape of a circle witha radius of 4 meters based on a noise level (e.g., decibel reading)emitted by machinery AD when operational. Based on the user beinglocated within the multiple crowdsourced user defined risk areas,surrounding assessment program 110 determines that machinery AD poses amoderate risk to the user at the determined location of the user.

Surrounding assessment program 110 receives (210) sensory data for theuser. In this embodiment, surrounding assessment program 110 receivessensory data for the user from one or more device associated with theuser (e.g., AR device, client device, fitness tracker). The sensory datais collected by one or more sensors that include an accelerometer, abarometer, a gyroscope sensor, a heart rate sensor, and an orientationsensor. Surrounding assessment program 110 utilizes the sensory datafrom an accelerometer to determine if the user is stationary, walking,or running. Surrounding assessment program 110 utilizes the sensory datafrom a barometer to determine a pressure at the determine location forthe user. Surrounding assessment program 110 utilizes sensory data froma gyroscope sensor to determine if the user is upright, leaning forward,leaning backwards, and/or bending down. Surrounding assessment program110 utilizes sensory data from a heart rate sensor to determine if theuser is experiencing elevated heart rate (e.g., due to loud machinery)or lowered heart rate (e.g., due to drowsiness). Surrounding assessmentprogram 110 utilizes an orientation sensor to determine a direction theuser is facing while stationary or moving. Surrounding assessmentprogram 110 utilizes the received sensory data to weight the riskfactors for the identified items in the surrounding area based on aspecific user.

In one example, surrounding assessment program 110 previously identifiedmachinery BA in a surrounding area of a user, as posing a low risk.Surrounding assessment program 110 receives sensory data for the userand determines an orientation of the user, where the orientation of theuser positions machinery BA behind the user. Furthermore, surroundingassessment program 110 determines the user is moving backwards towardsmachinery BA behind the user, where the user does not have visual ofmachinery BA due to machinery BA a position of machinery BA not being ina line of sight of the user. Surrounding assessment program 110 appliesa higher weight to the low risk assessment of machinery BA, since theuser is approaching machinery BA and does not have a visual of machineryBA.

In another example, surrounding assessment program 110 previouslyidentified machinery BB in a surrounding area of a user as posing amoderate risk, where machinery BB is a mobile item. Surroundingassessment program 110 receives sensory data for the user and determinesan orientation of the user, where the orientation of the user positionsmachinery BB behind a pilar on a warehouse floor and is not in a line ofsight of the user. Furthermore, surrounding assessment program 110determines the user is moving forward along a path that intersects witha set path of machinery BB, where a potential of a collision between theuser and machinery BB exists. surrounding assessment program 110 appliesa higher weight to the moderate risk assessment, since the user isapproaching machinery BB, does not have a visual of machinery BB, and apotential of a collision exists.

Surrounding assessment program 110 generates (212) a heat map for thesurrounding area of the user. In this embodiment, surrounding assessmentprogram 110 generates the heat map for the surrounding area of the user,where the surrounding area represents a surrounding vicinity of the user(e.g., radius of 3 meters). In another embodiment, surroundingassessment program 110 generates a first heat map for an area in a lineof sight as captured by a forward-facing camera on an AR device andgenerates a second heat map for a surrounding vicinity of the user,where the first heat map can be three-dimensional, and the second heatmap can be two dimensional. Surrounding assessment program 110 generatesthe heat map by applying the weighted risk levels to the identifieditems in the surrounding area of the user, where each weighted risklevel is associated with each identified item in the surrounding area ofthe user. The heat map represents a data visualization technique thatshow a magnitude of risk level as color in two or three dimensions,where a variation in a hue and/or intensity of a color represents therisk level for each of the identified items in the surrounding area ofthe user.

Surrounding assessment program 110 displays (214) the heat map for thesurrounding area. Surrounding assessment program 110 displays the heatmap for the surrounding area in one or more devices associated with theuser (e.g., mobile device, AR device). For displaying the heat map forthe surrounding area in an AR device, surrounding assessment program 110displays a three-dimensional overlay of the heat map on a portion of theidentified items in the line of sight of the user, where eachthree-dimensional overlay on each identified item represents a weightedrisk level to the user. As a user rotates, the AR device captures anorientation of the user and surrounding assessment program 110 displaysthe three-dimension overlay of the heat map on another portion of theidentified items in the new line of sight of the user. Surroundingassessment program 110 can also display multiple overlays if a firstidentified item blocks a view of a second identified item, where thefirst identified item has a weighted risk level different from a weightrisk level of the second identified item. Surrounding assessment program110 can display an overlay for the first identified item on top of anoverlay for the second identified item, representing that the firstidentified item blocks a view of the second identified item. Surroundingassessment program 110 can also display side markers (discussed infurther detail with regard to FIG. 3A) in a peripheral area of a view inthe AR device, where each side marker is associated with an identifieditem in the surrounding area that is not located in a line of sight ofthe user. Each side marker indicates a position of an identified item inthe surrounding area and as the user changes an orientation of the ARdevice, the identified item comes into view in a line of sight of theuser and surrounding assessment program 110 ceases to display thecorresponding side marker.

For displaying the heat map for a surrounding area in a client device(e.g., mobile device), surrounding assessment program 110 displays atwo-dimensional overlay on an overhead view of the surrounding area ofthe user at the determined location. Surrounding assessment program 110allows for the user to view an overhead view for the line of sight ofthe AR device associated with the user or an overhead view for thevicinity surrounding the user at the determined location. Thetwo-dimensional overlay on the overhead view allows for the user tolocate any identified items that are blocked by other identified itemsin the line of sight view in the AR device. For stationary identifieditems, surrounding assessment program 110 overlays the heat map on thestationary identified items. For mobile identified items, surroundingassessment program 110 overlays the heat map on the mobile identifieditems, where the overlay follows an expected movement of the mobileidentified item as it travels along a known set path.

Surrounding assessment program 110 determines (216) a path through theheat map for the surrounding area of the user. Surrounding assessmentprogram 110 utilizes previously received historical information todetermine the path through the heat map for the surrounding area of theuser to avoid the identified items in the surrounding area. Aspreviously discussed, the historical information includes a location forspecific machinery in the determined location, a list of manufacturersspecified risks for specific machinery, specified walking areas,specified risk areas, historically identified walking areas, locationsfor incident reports, and crowdsourced user defined risk areas. Thelocation for specific machinery in the determined location can include alocation of an area for a footprint of the specific machinery,dimensions for the specific machinery, an operational area for thespecific machinery, and an operational volume for the specificmachinery. Surrounding assessment program 110 utilizes the previouslyreceived historical information to establish paths through the heat map,if the user is not interacting with an identified item. A distancebetween a path and a weighted high-risk item can be greater than adistance between a path and a weighted low risk item in the surroundingarea of the user, thus allowing a greater margin of safety for the userwith the AR device.

Surrounding assessment program 110 displays (218) the path through theheat map for the surrounding area. Surrounding assessment program 110displays the path through the heat map for the surrounding area in oneor more device associated with the user (e.g., mobile device, ARdevice). For displaying the path through the heat map for thesurrounding area in an AR device, surrounding assessment program 110displays a three-dimensional outline overlay on a determined path fornavigating between the identified items in the heat map for thesurrounding area. Additionally, surrounding assessment program 110displays three-dimensional directional arrows indicating possible pathsto take along the outline overlay for the determined path for navigatingbetween the identified items in the heat map for the surrounding area.For displaying the path through the heat map for the surrounding area ina client device (e.g., mobile device), displays a two-dimensionaloutline overlay on an overhead view of a determined path for navigatingbetween the identified items in the heat map for the surrounding area.Additionally, surrounding assessment program 110 displaystwo-dimensional directional arrows indicating possible paths to takealong the outline overlay for the determined path for navigating betweenthe identified item in the heat map for the surrounding area.

FIG. 3A illustrates a first-person perspective view of a heat mapvisualization that a surrounding assessment program generates for asurrounding area, in accordance with an embodiment of the presentinvention. In this example, surrounding assessment program 110 displaysa heat map for a surrounding area of a user in an AR device, wheresurrounding assessment program 110 displays a three-dimensional overlayof the heat map on a portion of identified item in a line of sight ofthe user wearing the AR device. Surrounding assessment program 110previously identified items 302, 304, and 306 in the surrounding area ofthe user, where identified item 302 is low risk, identified item 304 ishigh risk and identified item 306 is moderate risk. Surroundingassessment program 110 also displays side markers 308 and 310 in leftperipheral area 312 and side markers 314 and 316 in right peripheralarea 318. Side marker 308 indicates there is an identified objected tothe left of the user, outside of the line of sight view in the ARdevice. Side marker 310 indicates there is an identified objected behindthe user, outside of the line of sight view in the AR device. Sidemarkers 314 and 316 indicates there are two identified objected to theright of the user, outside of the line of sight view in the AR device.If the user rotates in the left direction, the identified object to theleft of the user comes into view in the line of sight view of the ARdevice and surrounding assessment program 110 no longer needs to displayside marker 308 in left peripheral area 312. Surrounding assessmentprogram 110 also displays a three-dimensional outline overlay 320 on adetermined path for navigating between identified items 302, 304, and306 and three-dimensional directional arrows indicating possible pathsto take along the three-dimensional outline overlay 320 for thedetermined path.

FIG. 3B illustrates an overhead view of a heat map visualization that asurrounding assessment program generates for a surrounding area, inaccordance with an embodiment of the present invention. In this example,surrounding assessment program 110 displays a heat map for a surroundingarea of a user in a mobile device, where surrounding assessment program110 displays a two-dimensional overlay of the heat map on an overheadview on a portion of identified item in a line of sight of the userwearing the AR device. As previously mentioned in FIG. 3A, surroundingassessment program 110 previously identified items 302, 304, and 306 inthe surrounding area of the user, where identified item 302 is low risk,identified item 304 is high risk and identified item 306 is moderaterisk. Surrounding assessment program 110 also displays a two-dimensionaloutline overlay 320 on a determined path for navigating betweenidentified items 302, 304, and 306 and three-dimensional directionalarrows indicating possible paths to take along the three-dimensionaloutline overlay 320 for the determined path. Surrounding assessmentprogram 110 provides an overhead view of the surrounding area in theline of sight of the user to allow for the user to view areas blocked byidentified items 302, 304, and 306.

Embodiments of the present invention provide a system for analyzingsurrounding visual inputs, contextual inputs, and IoT device feed from asurrounding environment to identify types of risk, severity of risks,and a distance from target objects which include the risks. The systemgenerates a heat map of risk factors within the surroundings of the userand displays a safe movement path for the user. While showing the heatmap of risk factors, the system considers a state of a user,qualifications of a user for operating a target object, and geolocationinformation and customizes the heat map of risk factors accordingly forthe user. The system can also utilize a knowledge corpus to show if anyrisk factor is time or location dependent, where a target object canhave different levels of risk factors at different times of a day, week,month, and season. The system can share risk insights based on an opt-inagreement and aggregated crowdsourcing experiences, where users canprovide feedback regarding experiences of risk in the surroundingenvironment. The system can predict a mobility path and speed ofmobility of target objects to adjust the safe movement path for the userand display the safe movement path to the user in the surroundingenvironment.

Embodiments of the present invention utilizes historical learning byidentifying risk factors associated with different objects andcategorizing the different objects. Levels of risk can be calculatedbased on distance, historical evidence, or event occurrences withregards to the different objects. Types of risk are identified utilizingK-means/density-based spatial clustering of application with noise(DBSCAN) clustering infused with a long-short term memory network (LSTM)and convolutional neural network (CNN) model for tracking historicalevidence and event occurrences, for alerting a user via a heat mapdisplay. Utilizing historical data, personalized profile preferences,and qualification required to address the risk are identified andfactored into the weights of the algorithm. IoT data, sensor imaging,visuals, and audio are gathered from a surrounding area to identity rickfactors and a set of spatial temporal interest points are extracted froma capturing device. At each interest point, extracting a histogram ofoptical flow (HoF) feature computed on a three-dimensional space-timevolume from frame buffers being captured from the visual device embeddedin an AR device. A randomly sampled set of motion descriptors fromprevious recordings and current capture frame is clustered utilizingK-means to form a “visual codebook”. A final “bag of visual words andobjects” representing frames includes a vector of k values, where thei^(th) value represents a number of motion descriptors in the framesthat belong to the i^(th) cluster.

For the equation provided below, features utilized by the objectdetector are denoted as F and a probability of an activity A_(i) giventhe object features is computed by applying a chain rule as follows:

${P( A_{i} \middle| F_{o} )} = {\sum\limits_{j = 1}^{❘O❘}{{P( A_{i} \middle| O_{j} )}*{P( O_{j} \middle| F_{o} )}}}$

Where |O| is the total number of object detectors. User's reactionarybehavior is incorporated and correlated with a Pearson's correlation inorder to classify the criticality of objects detected in further framesof consideration, which can be clustered into multiple risk categories(e.g., low (0-30), moderate (30-70), and high (70-100)).

Embodiments of the present invention create a knowledge corpus for acentralized information repository associated with each user. Over timeand training the knowledge corpus is expanded with additionalinformation relating to the user. Optional opt-in embodiments can permita utilization of a crowdsourced knowledge corpus in conjunction with theknowledge corpus for the centralized information repository associatedwith each user. The system can draw user context and personal traitidentifier from the knowledge corpus to join with environmentalsurroundings. The system can leverage edge computing to recognize anobject and types of risk associated with the identified object based onlocation and an object type. The system can display a heat map for asurrounding area as a user travels within the environment and a safemovement path for the user is displayed and/or communicated (e.g., audioinstructions) to the user.

FIG. 4 depicts computer system 400, where server computer 102 is anexample of a computer system 400 that includes surrounding assessmentprogram 110. The computer system includes processors 404, cache 416,memory 406, persistent storage 408, communications unit 410,input/output (I/O) interface(s) 412 and communications fabric 402.Communications fabric 402 provides communications between cache 416,memory 406, persistent storage 408, communications unit 410, andinput/output (I/O) interface(s) 412. Communications fabric 402 can beimplemented with any architecture designed for passing data and/orcontrol information between processors (such as microprocessors,communications and network processors, etc.), system memory, peripheraldevices, and any other hardware components within a system. For example,communications fabric 402 can be implemented with one or more buses or acrossbar switch.

Memory 406 and persistent storage 408 are computer readable storagemedia. In this embodiment, memory 406 includes random access memory(RAM). In general, memory 406 can include any suitable volatile ornon-volatile computer readable storage media. Cache 416 is a fast memorythat enhances the performance of processors 404 by holding recentlyaccessed data, and data near recently accessed data, from memory 406.

Program instructions and data used to practice embodiments of thepresent invention may be stored in persistent storage 408 and in memory406 for execution by one or more of the respective processors 404 viacache 416. In an embodiment, persistent storage 408 includes a magnetichard disk drive. Alternatively, or in addition to a magnetic hard diskdrive, persistent storage 408 can include a solid state hard drive, asemiconductor storage device, read-only memory (ROM), erasableprogrammable read-only memory (EPROM), flash memory, or any othercomputer readable storage media that is capable of storing programinstructions or digital information.

The media used by persistent storage 408 may also be removable. Forexample, a removable hard drive may be used for persistent storage 408.Other examples include optical and magnetic disks, thumb drives, andsmart cards that are inserted into a drive for transfer onto anothercomputer readable storage medium that is also part of persistent storage408.

Communications unit 410, in these examples, provides for communicationswith other data processing systems or devices. In these examples,communications unit 410 includes one or more network interface cards.Communications unit 410 may provide communications through the use ofeither or both physical and wireless communications links. Programinstructions and data used to practice embodiments of the presentinvention may be downloaded to persistent storage 408 throughcommunications unit 410.

I/O interface(s) 412 allows for input and output of data with otherdevices that may be connected to each computer system. For example, I/Ointerface 412 may provide a connection to external devices 418 such as akeyboard, keypad, a touch screen, and/or some other suitable inputdevice. External devices 418 can also include portable computer readablestorage media such as, for example, thumb drives, portable optical ormagnetic disks, and memory cards. Software and data used to practiceembodiments of the present invention can be stored on such portablecomputer readable storage media and can be loaded onto persistentstorage 408 via I/O interface(s) 412. I/O interface(s) 412 also connectto display 420.

Display 420 provides a mechanism to display data to a user and may be,for example, a computer monitor.

The programs described herein are identified based upon the applicationfor which they are implemented in a specific embodiment of theinvention. However, it should be appreciated that any particular programnomenclature herein is used merely for convenience, and thus theinvention should not be limited to use solely in any specificapplication identified and/or implied by such nomenclature.

The present invention may be a system, a method, and/or a computerprogram product at any possible technical detail level of integration.The computer program product may include a computer readable storagemedium (or media) having computer readable program instructions thereonfor causing a processor to carry out aspects of the present invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, configuration data for integrated circuitry, oreither source code or object code written in any combination of one ormore programming languages, including an object oriented programminglanguage such as Smalltalk, C++, or the like, and procedural programminglanguages, such as the “C” programming language or similar programminglanguages. The computer readable program instructions may executeentirely on the user's computer, partly on the user's computer, as astand-alone software package, partly on the user's computer and partlyon a remote computer or entirely on the remote computer or server. Inthe latter scenario, the remote computer may be connected to the user'scomputer through any type of network, including a local area network(LAN) or a wide area network (WAN), or the connection may be made to anexternal computer (for example, through the Internet using an InternetService Provider). In some embodiments, electronic circuitry including,for example, programmable logic circuitry, field-programmable gatearrays (FPGA), or programmable logic arrays (PLA) may execute thecomputer readable program instructions by utilizing state information ofthe computer readable program instructions to personalize the electroniccircuitry, in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a computer, or other programmable data processing apparatusto produce a machine, such that the instructions, which execute via theprocessor of the computer or other programmable data processingapparatus, create means for implementing the functions/acts specified inthe flowchart and/or block diagram block or blocks. These computerreadable program instructions may also be stored in a computer readablestorage medium that can direct a computer, a programmable dataprocessing apparatus, and/or other devices to function in a particularmanner, such that the computer readable storage medium havinginstructions stored therein comprises an article of manufactureincluding instructions which implement aspects of the function/actspecified in the flowchart and/or block diagram block or blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the blocks may occur out of theorder noted in the Figures. For example, two blocks shown in successionmay, in fact, be accomplished as one step, executed concurrently,substantially concurrently, in a partially or wholly temporallyoverlapping manner, or the blocks may sometimes be executed in thereverse order, depending upon the functionality involved. It will alsobe noted that each block of the block diagrams and/or flowchartillustration, and combinations of blocks in the block diagrams and/orflowchart illustration, can be implemented by special purposehardware-based systems that perform the specified functions or acts orcarry out combinations of special purpose hardware and computerinstructions.

It is to be understood that although this disclosure includes a detaileddescription on cloud computing, implementation of the teachings recitedherein are not limited to a cloud computing environment. Rather,embodiments of the present invention are capable of being implemented inconjunction with any other type of computing environment now known orlater developed.

Cloud computing is a model of service delivery for enabling convenient,on-demand network access to a shared pool of configurable computingresources (e.g., networks, network bandwidth, servers, processing,memory, storage, applications, virtual machines, and services) that canbe rapidly provisioned and released with minimal management effort orinteraction with a provider of the service. This cloud model may includeat least five characteristics, at least three service models, and atleast four deployment models.

Characteristics are as follows:

On-demand self-service: a cloud consumer can unilaterally provisioncomputing capabilities, such as server time and network storage, asneeded automatically without requiring human interaction with theservice's provider.

Broad network access: capabilities are available over a network andaccessed through standard mechanisms that promote use by heterogeneousthin or thick client platforms (e.g., mobile phones, laptops, and PDAs).

Resource pooling: the provider's computing resources are pooled to servemultiple consumers using a multi-tenant model, with different physicaland virtual resources dynamically assigned and reassigned according todemand. There is a sense of location independence in that the consumergenerally has no control or knowledge over the exact location of theprovided resources but may be able to specify location at a higher levelof abstraction (e.g., country, state, or datacenter).

Rapid elasticity: capabilities can be rapidly and elasticallyprovisioned, in some cases automatically, to quickly scale out andrapidly released to quickly scale in. To the consumer, the capabilitiesavailable for provisioning often appear to be unlimited and can bepurchased in any quantity at any time.

Measured service: cloud systems automatically control and optimizeresource use by leveraging a metering capability at some level ofabstraction appropriate to the type of service (e.g., storage,processing, bandwidth, and active user accounts). Resource usage can bemonitored, controlled, and reported, providing transparency for both theprovider and consumer of the utilized service.

Service Models are as follows:

Software as a Service (SaaS): the capability provided to the consumer isto use the provider's applications running on a cloud infrastructure.The applications are accessible from various client devices through athin client interface such as a web browser (e.g., web-based e-mail).The consumer does not manage or control the underlying cloudinfrastructure including network, servers, operating systems, storage,or even individual application capabilities, with the possible exceptionof limited user-specific application configuration settings.

Platform as a Service (PaaS): the capability provided to the consumer isto deploy onto the cloud infrastructure consumer-created or acquiredapplications created using programming languages and tools supported bythe provider. The consumer does not manage or control the underlyingcloud infrastructure including networks, servers, operating systems, orstorage, but has control over the deployed applications and possiblyapplication hosting environment configurations.

Infrastructure as a Service (IaaS): the capability provided to theconsumer is to provision processing, storage, networks, and otherfundamental computing resources where the consumer is able to deploy andrun arbitrary software, which can include operating systems andapplications. The consumer does not manage or control the underlyingcloud infrastructure but has control over operating systems, storage,deployed applications, and possibly limited control of select networkingcomponents (e.g., host firewalls).

Deployment Models are as follows:

Private cloud: the cloud infrastructure is operated solely for anorganization. It may be managed by the organization or a third party andmay exist on-premises or off-premises.

Community cloud: the cloud infrastructure is shared by severalorganizations and supports a specific community that has shared concerns(e.g., mission, security requirements, policy, and complianceconsiderations). It may be managed by the organizations or a third partyand may exist on-premises or off-premises.

Public cloud: the cloud infrastructure is made available to the generalpublic or a large industry group and is owned by an organization sellingcloud services.

Hybrid cloud: the cloud infrastructure is a composition of two or moreclouds (private, community, or public) that remain unique entities butare bound together by standardized or proprietary technology thatenables data and application portability (e.g., cloud bursting forload-balancing between clouds).

A cloud computing environment is service oriented with a focus onstatelessness, low coupling, modularity, and semantic interoperability.At the heart of cloud computing is an infrastructure that includes anetwork of interconnected nodes.

Referring now to FIG. 5, illustrative cloud computing environment 50 isdepicted. As shown, cloud computing environment 50 includes one or morecloud computing nodes 10 with which local computing devices used bycloud consumers, such as, for example, personal digital assistant (PDA)or cellular telephone 54A, desktop computer 54B, laptop computer 54C,and/or automobile computer system 54N may communicate. Nodes 10 maycommunicate with one another. They may be grouped (not shown) physicallyor virtually, in one or more networks, such as Private, Community,Public, or Hybrid clouds as described hereinabove, or a combinationthereof. This allows cloud computing environment 50 to offerinfrastructure, platforms and/or software as services for which a cloudconsumer does not need to maintain resources on a local computingdevice. It is understood that the types of computing devices 54A-N shownin FIG. 5 are intended to be illustrative only and that computing nodes10 and cloud computing environment 50 can communicate with any type ofcomputerized device over any type of network and/or network addressableconnection (e.g., using a web browser).

Referring now to FIG. 6, a set of functional abstraction layers providedby cloud computing environment 50 (FIG. 5) is shown. It should beunderstood in advance that the components, layers, and functions shownin FIG. 6 are intended to be illustrative only and embodiments of theinvention are not limited thereto. As depicted, the following layers andcorresponding functions are provided:

Hardware and software layer 60 include hardware and software components.Examples of hardware components include: mainframes 61; RISC (ReducedInstruction Set Computer) architecture based servers 62; servers 63;blade servers 64; storage devices 65; and networks and networkingcomponents 66. In some embodiments, software components include networkapplication server software 67 and database software 68.

Virtualization layer 70 provides an abstraction layer from which thefollowing examples of virtual entities may be provided: virtual servers71; virtual storage 72; virtual networks 73, including virtual privatenetworks; virtual applications and operating systems 74; and virtualclients 75.

In one example, management layer 80 may provide the functions describedbelow. Resource provisioning 81 provides dynamic procurement ofcomputing resources and other resources that are utilized to performtasks within the cloud computing environment. Metering and Pricing 82provide cost tracking as resources are utilized within the cloudcomputing environment, and billing or invoicing for consumption of theseresources. In one example, these resources may include applicationsoftware licenses. Security provides identity verification for cloudconsumers and tasks, as well as protection for data and other resources.User portal 83 provides access to the cloud computing environment forconsumers and system administrators. Service level management 84provides cloud computing resource allocation and management such thatrequired service levels are met. Service Level Agreement (SLA) planningand fulfillment 85 provide pre-arrangement for, and procurement of,cloud computing resources for which a future requirement is anticipatedin accordance with an SLA.

Workloads layer 90 provides examples of functionality for which thecloud computing environment may be utilized. Examples of workloads andfunctions which may be provided from this layer include: mapping andnavigation 91; software development and lifecycle management 92; virtualclassroom education delivery 93; data analytics processing 94;transaction processing 95; and surrounding assessment program 110.

The programs described herein are identified based upon the applicationfor which they are implemented in a specific embodiment of theinvention. However, it should be appreciated that any particular programnomenclature herein is used merely for convenience, and thus theinvention should not be limited to use solely in any specificapplication identified and/or implied by such nomenclature.

The present invention may be a system, a method, and/or a computerprogram product at any possible technical detail level of integration.The computer program product may include a computer readable storagemedium (or media) having computer readable program instructions thereonfor causing a processor to carry out aspects of the present invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, configuration data for integrated circuitry, oreither source code or object code written in any combination of one ormore programming languages, including an object oriented programminglanguage such as Smalltalk, C++, or the like, and procedural programminglanguages, such as the “C” programming language or similar programminglanguages. The computer readable program instructions may executeentirely on the user's computer, partly on the user's computer, as astand-alone software package, partly on the user's computer and partlyon a remote computer or entirely on the remote computer or server. Inthe latter scenario, the remote computer may be connected to the user'scomputer through any type of network, including a local area network(LAN) or a wide area network (WAN), or the connection may be made to anexternal computer (for example, through the Internet using an InternetService Provider). In some embodiments, electronic circuitry including,for example, programmable logic circuitry, field-programmable gatearrays (FPGA), or programmable logic arrays (PLA) may execute thecomputer readable program instructions by utilizing state information ofthe computer readable program instructions to personalize the electroniccircuitry, in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a computer, or other programmable data processing apparatusto produce a machine, such that the instructions, which execute via theprocessor of the computer or other programmable data processingapparatus, create means for implementing the functions/acts specified inthe flowchart and/or block diagram block or blocks. These computerreadable program instructions may also be stored in a computer readablestorage medium that can direct a computer, a programmable dataprocessing apparatus, and/or other devices to function in a particularmanner, such that the computer readable storage medium havinginstructions stored therein comprises an article of manufactureincluding instructions which implement aspects of the function/actspecified in the flowchart and/or block diagram block or blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the blocks may occur out of theorder noted in the Figures. For example, two blocks shown in successionmay, in fact, be accomplished as one step, executed concurrently,substantially concurrently, in a partially or wholly temporallyoverlapping manner, or the blocks may sometimes be executed in thereverse order, depending upon the functionality involved. It will alsobe noted that each block of the block diagrams and/or flowchartillustration, and combinations of blocks in the block diagrams and/orflowchart illustration, can be implemented by special purposehardware-based systems that perform the specified functions or acts orcarry out combinations of special purpose hardware and computerinstructions.

What is claimed is:
 1. A computer-implemented method comprising:responsive to determining a location of a user, receiving historicalinformation for the location of the user; identifying a plurality ofitems in a surrounding area of the user, wherein the surrounding areaincludes at least a line of sight of the user as captured by anaugmented reality device associated with the user; analyzing thesurrounding area of the user based on the plurality of items and thehistorical information for the location of the user to produce aplurality of risk levels for the plurality of items, wherein each risklevel from the plurality of risk levels is associated with each itemfrom the plurality of items; responsive to receiving sensory data forthe user, generating a heat map for the surrounding area of the user,wherein the heat map is based on a plurality of risk factors for theplurality of items in the surrounding area; and displaying the heat mapto the user in the augmented reality device.
 2. The computer-implementedmethod of claim 1, further comprising: determining a path through theheat map for the surrounding area; and displaying the path through theheat map for the surrounding area in the augmented reality device,wherein the path is displayed as a first overlay in the line of sight ofthe user as captured by the augmented reality device.
 3. Thecomputer-implemented method of claim 2, wherein the heat map isdisplayed as a second overlay on each of the plurality of items in theline of sight of the user as captured by the augmented reality device.4. The computer-implemented method of claim 1, wherein the analyzing thesurrounding area of the user, further comprises: determining eachdistance from a plurality of distances between the user and each itemfrom the plurality of items; parsing the received historical informationfor the location to identify each portion of the historical informationassociated with each item from the plurality of items; and determining arisk score for each item from the plurality of items relative to thelocation of the user, wherein the risk score is associated with aspecific risk level from the plurality of risk levels.
 5. Thecomputer-implemented method of claim 1, further comprising: receivingsensory data for the user as captured by one or more sensors, whereinthe one or more sensors are selected from a group consisting of: anaccelerometer, a barometer, a gyroscope sensor, a heart rate sensor, andan orientation sensor.
 6. The computer-implemented method of claim 1,wherein the historical information includes operational qualificationsfor operating one or more items from the plurality of items by the userat the location.
 7. The computer-implemented method of claim 6, whereinthe historical information further includes a location for a specificitem in the determined location, a list of manufacturers specified risksfor the specific item, specified walking areas, specified risk areas,historically identified walking areas, locations for incident reports,and crowdsourced user defined risk areas.
 8. A computer program productcomprising one or more computer readable storage media and programinstructions collectively stored on the one or more computer readablestorage media, the stored program instructions executable by one or morecomputer processors, the stored program instructions comprising: programinstructions to, responsive to determining a location of a user, receivehistorical information for the location of the user; programinstructions to identify a plurality of items in a surrounding area ofthe user, wherein the surrounding area includes at least a line of sightof the user as captured by an augmented reality device associated withthe user; program instructions to analyze the surrounding area of theuser based on the plurality of items and the historical information forthe location of the user to produce a plurality of risk levels for theplurality of items, wherein each risk level from the plurality of risklevels is associated with each item from the plurality of items; programinstructions to, responsive to receiving sensory data for the user,generate a heat map for the surrounding area of the user, wherein theheat map is based on a plurality of risk factors for the plurality ofitems in the surrounding area; and program instructions to display theheat map to the user in the augmented reality device.
 9. The computerprogram product of claim 8, the stored program instructions furthercomprising: program instructions to determine a path through the heatmap for the surrounding area; and program instructions to display thepath through the heat map for the surrounding area in the augmentedreality device, wherein the path is displayed as a first overlay in theline of sight of the user as captured by the augmented reality device.10. The computer program product of claim 9, wherein the heat map isdisplayed as a second overlay on each of the plurality of items in theline of sight of the user as captured by the augmented reality device.11. The computer program product of claim 8, wherein the analyzing thesurrounding area of the user, the stored program instructions furthercomprises: program instructions to determine each distance from aplurality of distances between the user and each item from the pluralityof items; program instructions to parse the received historicalinformation for the location to identify each portion of the historicalinformation associated with each item from the plurality of items; andprogram instructions to determine a risk score for each item from theplurality of items relative to the location of the user, wherein therisk score is associated with a specific risk level from the pluralityof risk levels.
 12. The computer program product of claim 8, the storedprogram instructions further comprising: program instructions to receivesensory data for the user as captured by one or more sensors, whereinthe one or more sensors are selected from a group consisting of: anaccelerometer, a barometer, a gyroscope sensor, a heart rate sensor, andan orientation sensor.
 13. The computer program product of claim 8,wherein the historical information includes operational qualificationsfor operating one or more items from the plurality of items by the userat the location.
 14. The computer program product of claim 13, whereinthe historical information further includes a location for a specificitem in the determined location, a list of manufacturers specified risksfor the specific item, specified walking areas, specified risk areas,historically identified walking areas, locations for incident reports,and crowdsourced user defined risk areas.
 15. A computer systemcomprising: one or more computer processors; one or more computerreadable storage media; and program instructions stored on the computerreadable storage media for execution by at least one of the one or morecomputer processors, the program instructions comprising: programinstructions to, responsive to determining a location of a user, receivehistorical information for the location of the user; programinstructions to identify a plurality of items in a surrounding area ofthe user, wherein the surrounding area includes at least a line of sightof the user as captured by an augmented reality device associated withthe user; program instructions to analyze the surrounding area of theuser based on the plurality of items and the historical information forthe location of the user to produce a plurality of risk levels for theplurality of items, wherein each risk level from the plurality of risklevels is associated with each item from the plurality of items; programinstructions to, responsive to receiving sensory data for the user,generate a heat map for the surrounding area of the user, wherein theheat map is based on a plurality of risk factors for the plurality ofitems in the surrounding area; and program instructions to display theheat map to the user in the augmented reality device.
 16. The computersystem of claim 15, the stored program instructions further comprising:program instructions to determine a path through the heat map for thesurrounding area; and program instructions to display the path throughthe heat map for the surrounding area in the augmented reality device,wherein the path is displayed as a first overlay in the line of sight ofthe user as captured by the augmented reality device.
 17. The computersystem of claim 16, wherein the heat map is displayed as a secondoverlay on each of the plurality of items in the line of sight of theuser as captured by the augmented reality device.
 18. The computersystem of claim 15, wherein the analyzing the surrounding area of theuser, the stored program instructions further comprises: programinstructions to determine each distance from a plurality of distancesbetween the user and each item from the plurality of items; programinstructions to parse the received historical information for thelocation to identify each portion of the historical informationassociated with each item from the plurality of items; and programinstructions to determine a risk score for each item from the pluralityof items relative to the location of the user, wherein the risk score isassociated with a specific risk level from the plurality of risk levels.19. The computer system of claim 15, the stored program instructionsfurther comprising: program instructions to receive sensory data for theuser as captured by one or more sensors, wherein the one or more sensorsare selected from a group consisting of: an accelerometer, a barometer,a gyroscope sensor, a heart rate sensor, and an orientation sensor. 20.The computer system of claim 15, wherein the historical informationincludes a location for a specific item in the determined location, alist of manufacturers specified risks for the specific item, specifiedwalking areas, specified risk areas, historically identified walkingareas, locations for incident reports, and crowdsourced user definedrisk areas.